1. Field of the Invention
The present invention relates to a visual axis detecting method for detecting the observer's visual axis, a visual axis detecting means, and an image device having the visual axis detecting means, for example, a camera such as a still camera or a video camera, a display device such as a goggle type display, etc.
2. Related Background Art
Many proposals have been made on the technology to facilitate interfaces between various devices and a man by detecting the direction of the visual axis of the man. Among them, the detection of visual axis is relatively easy with an apparatus such as image devices constructed in such a manner that a man looks into a display screen.
For example, Japanese Laid-open Patent Application No. 1-241511 discloses the invention concerning a camera provided with a visual axis detecting apparatus, which is recently brought into actual use. FIG. 13 shows a schematic diagram of the visual axis detecting apparatus as described in the above application.
In FIG. 13, reference numeral 1 designates an objective lens, which is represented by a single lens for convenience' sake but which is actually constructed of a number of lenses, as well known. Numeral 2 denotes a main mirror, which is obliquely set in or is withdrawn from a photo-taking optical path, depending upon whether the camera is in an observing state or in a photo-taking state. Numeral 3 denotes a submirror, which reflects a beam of light having passed through the main mirror 2 down toward the camera body not shown. Further, 4a is a shutter, 4b a stop disposed in the objective lens 1, and 4c a driving mechanism for moving the objective lens 1 along the optical axis for focusing.
Numeral 5 represents a photosensitive member, which is a silver-salt film, a solid state image sensing device such as a CCD or MOS image sensor, or a camera tube such as a vidicon.
Further, 6a is a focus detecting unit.
There is an eyepiece lens 9 disposed behind the exit plane of a pentagonal roof prism 8 for changing the finder optical path, which is used to observe a focus plate 7 by the observer's eye 15. Numeral 10 represents an optical splitter, which is, for example, a dichroic mirror for reflecting infrared light and which is disposed in the eyepiece lens 9. Numeral 11 denotes a condenser lens, 12 an optical splitter such as a half mirror, and 13 an illumination light source such as LED, which preferably emits infrared light (and near infrared light). A beam of light emitted from the infrared illumination light source 13 is changed into a beam of parallel rays advancing along the finder optical path by the power of the condenser lens 11 and the rear surface (observer-side surface) of the eyepiece lens 9. Numeral 14 is a photoelectric converter.
Light having passed through the objective lens 1 is reflected by the main mirror 2 to pass through the focus plate 7 and then to repeat reflection inside the pentagonal roof prism 8. After that, the light enters the observer's eyeball 15 looking into the eyepiece lens 9.
Since surfaces in the human eye have such changes of refractive index as shown in FIG. 14, the illumination light is reflected with different intensities depending upon a magnitude of index change. That is, the light is reflected with intensities decreasing in the order of the front surface of the cornea, the front surface and rear surface of the eye lens, and the rear surface of the cornea. It is also seen from results of trace of paraxial rays that with respect to the front of the eyeball, positions of reflected images from the respective surfaces with incidence of parallel rays are as shown in FIG. 15. These images are called Purkinje images, which are numbered in order from the front surface of the cornea as a first Purkinje image, a second Purkinje image, . . . . As apparent from FIG. 15, the three Purkinje images excluding the third image are concentrated immediately after the third surface, i.e., the front surface of the eye lens, and from the study on the index change as described, the reflection images have the respective intensities decreasing in the order of the first image, the fourth image, and the second image. Since the illumination light forming these images is in the infrared wave region, it is out of the range of sensation of the human eye, thus causing no trouble in observation of finder image. For this purpose, the wavelength of the illumination light is preferably longer than 700 nm, and more preferably longer than 750 nm, which is never sensed by any human eyes, regardless of differences between individuals.
The light reflected by the observer's eye travels backward in the path via the mirror 10 and the lens 11 then to be reflected by the half mirror 12 and thereafter to be received by the photoelectric converter 14. It is preferred that a filter for cutting the visible light but transmitting the infrared light be set in the optical path after the reflected light is separated from the finder optical path and before it is received by the photoelectric converter. The filter is for cutting the reflection light from the cornea, of the visible light of the finder image, and for photoelectrically converting only reflection light of the infrared illumination light, which is significant as light signals, into electric signals. A photoelectric surface is located at such a position that the overall power of the lens 11 and the rear surface of eyepiece lens 9 forms an image of the vicinity of the front surface of the lens, i.e., an image of the vicinity of the pupil of the observer's eye. By this arrangement, the reflection light is received while the first, second, and fourth Purkinje images are focused, but the third Purkinje image, an amount of reflection light of which is not necessarily weak, is defocused so as to diffuse the light, which presents little contribution to photoelectric signals.
Next described is the principle of operation of the part of visual axis detecting apparatus in this example. In the apparatus of FIG. 13, the infrared illumination light source 13 is a point light source and the position of illumination point light source 13 is adjusted so as to emit light at the position of the screen center on the focus plate 7. In this case, when the optical axis of the observer's eyeball passes the screen center, the illumination light source is located on an extension line of the optical path of eyeball. Thus, the Purkinje images are aligned as point images on the optical axis of eyeball. FIG. 16A shows the appearance of the vicinity of the pupil of the eyeball as observed from the front thereof. In the drawing, numeral 41 designates the iris, 42 the pupil, and 43 the superimposed Purkinje images. The brightly illuminated iris is observed in a ring shape and a bright spot of the superimposed Purkinje images from the respective surfaces is observed at the center of the dark, circular pupil 42. On the other hand, when the eyeball rotates left or right to direct the visual axis in an offset direction, the illumination light is obliquely incident thereinto with respect to the optical axis of eyeball. Thus, the Purkinje images move from the pupil center to offset positions. Further, because directions and amounts of movement are different depending upon the reflection surfaces, a plurality of Purkinje images 43, 44, . . . are observed from the front. FIG. 16B shows this state. When the optical axis of the observer's eye is directed to a position further away from the screen center, the above tendency is further enhanced, as shown in FIG. 16C Also, when the observer's eye is directed in the opposite direction, the directions of movement of the Purkinje images are also reversed.
The direction of the visual axis can be detected by converting this motion into an electric signal and performing signal processing.
FIG. 17A shows an example of positional relation between the first Purkinje image 62, the fourth Purkinje image 63, the pupil 61, and a photoelectric converting element array 64 when only horizontal movement of the visual axis is detected, and FIG. 17B diagrammatically shows outputs from respective photoelectric converting elements. Higher outputs on both sides represent the iris, and signals 65, 66 are obtained corresponding to the first Purkinje image and the fourth Purkinje image, respectively, in the dark pupil portion. The pupil center is obtained as a middle point between edge portions 67 and 68. Then the direction of the visual axis can be calculated by obtaining a distance between the pupil center and the first or fourth Purkinje image.
If knowledge of vertical and horizontal changes are desired, a two-dimensional array of photoelectric elements, for example, as shown in FIG. 18, may be prepared, whereby coordinates of the first Purkinje image 62 can be determined from a number of column 71 and a number of row 72 in the photoelectric conversion unit.
The use of a photoelectric conversion unit with a two-dimensional array of photoelectric elements can expect development of applications in broad regions including not only cameras but also image devices.
The above conventional example, however, needed to use a two-dimensional area sensor as the photoelectric conversion unit for receiving a two-dimensional reflection image of the eyeball, thus increasing the production cost of the visual axis detecting apparatus. Further, the area sensor generally has the following problems:
(1) It has fixed pattern noise (FPN) which presents fluctuations of output in each bit in a dark state. Thus, there is a possibility of error detection due to FPN in reading weak reflection signals. PA1 (2) Since the size of each pixel must be decreased, a quantity of light is not always sufficient, which substantially lowers the sensitivity of light-receiving elements. Thus, there would be cases in which weak signals are hardly detected accurately.